Skip to content

[Ascend] Fix input param of do_bench_npu - #5451

Merged
huangyiqun merged 1 commit into
masterfrom
fix_do_bench_npu_input_20260813
Aug 14, 2026
Merged

[Ascend] Fix input param of do_bench_npu#5451
huangyiqun merged 1 commit into
masterfrom
fix_do_bench_npu_input_20260813

Conversation

@zhzhcookie

@zhzhcookie zhzhcookie commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

PR Category

OP Test

Type of Change

Bug Fix

Description

do_bench_npu requires the number of iterations as input, rather than a duration.
Use default value instead.

Issue

Progress

  • Change is properly reviewed (1 reviewer required, 2 recommended).
  • Change is responded to an issue.
  • Change is fully covered by a UT.

Performance

@zhzhcookie
zhzhcookie marked this pull request as draft August 13, 2026 09:34
@github-actions

This comment was marked as resolved.

Sign in to view
@zhzhcookie
zhzhcookie marked this pull request as ready for review August 13, 2026 09:36
@zhzhcookie

zhzhcookie commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

/test add:cann9

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Test Environment

Env Setting Env Setting
Time 2026-08-13 17:46:29 -> 2026-08-13 17:42:47 FlagGems 5.3.4.post1.dev63+g560ea150c+g560ea150c -> 5.3.4.post1.dev63+g560ea150c+g560ea150c
Arch aarch64 OS ubuntu 22.04
Python 3.11.15 Torch 2.10.0+cpu
FlagTree 0.6.0+ascend3.5 Triton 3.5.1
Vendor ascend Device npu

Test Result

Accuracy Result

Summary

Case Status Duration Total Passed Skipped Failed
Before Passed 143.43 sec 698 650 48 0
After Passed 175.67 sec 698 650 48 0

Skipped Cases Before

  • Reason

    Skipped: Issues [Ascend] NPU 不支持 complex32 dtype，相关用例需 skip #3267: Ascend NPU does not support complex32 dtype

    Test cases

    • tests/test_add.py::test_add_complex:complex:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[16,7,57,32,29]

Skipped Cases After

  • Reason

    Skipped: Issues [Ascend] NPU 不支持 complex32 dtype，相关用例需 skip #3267: Ascend NPU does not support complex32 dtype

    Test cases

    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[20,320,15]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[1024,1024]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[1]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex64:[1]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex32:[20,320,15]
    • tests/test_add.py::test_add_complex:int_tensor:torch.complex64:[1024,1024]
    • tests/test_add.py::test_add_complex:complex:torch.complex64:[16,128,64,60]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[16,7,57,32,29]
    • tests/test_add.py::test_add_complex:int_scalar:torch.complex32:[]
    • tests/test_add.py::test_add_complex:float_tensor:torch.complex32:[16,128,64,60]
    • tests/test_add.py::test_add_complex:complex:torch.complex32:[]

Performance Result

Summary

Case Status Duration Reason
Before Failed 58.94 seconds
Failed: normal_:../third_party/op-plugin/op_plugin/ops/opapi/NormalKernelNpuOpApi.cpp:53 NPU function error: call aclnnInplaceNormal failed, error code is 161002
[ERROR] 2026-08-13-17:46:20 (PID:1928957, Device:0, RankID:-1) ERR00100 PTA call acl api failed.
[PID: 1928957] 2026-08-13-17:46:20.401.270 AclNN_Parameter_Error(EZ1001): Tensor self not implemented for DT_COMPLEX64, should be in dtype support list [DT_FLOAT16,DT_FLOAT,DT_DOUBLE,DT_INT16,DT_INT32,DT_INT64,DT_INT8,DT_UINT8,DT_BOOL,DT_BFLOAT16,].
After Failed 70.87 seconds
Failed: normal_:../third_party/op-plugin/op_plugin/ops/opapi/NormalKernelNpuOpApi.cpp:53 NPU function error: call aclnnInplaceNormal failed, error code is 161002
[ERROR] 2026-08-13-17:42:39 (PID:1916148, Device:0, RankID:-1) ERR00100 PTA call acl api failed.
[PID: 1916148] 2026-08-13-17:42:39.313.456 AclNN_Parameter_Error(EZ1001): Tensor self not implemented for DT_COMPLEX64, should be in dtype support list [DT_FLOAT16,DT_FLOAT,DT_DOUBLE,DT_INT16,DT_INT32,DT_INT64,DT_INT8,DT_UINT8,DT_BOOL,DT_BFLOAT16,].

Benchmark data

No performance data collected!

📎 Download the full log here

@zhzhcookie
zhzhcookie marked this pull request as draft August 13, 2026 09:57
@zhzhcookie
zhzhcookie marked this pull request as ready for review August 14, 2026 07:44
@0x45f

0x45f commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

/test abs:cann9

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Test Environment

Env Setting Env Setting
Time 2026-08-14 16:27:28 -> 2026-08-14 16:25:34 FlagGems 5.3.4.post1.dev63+g560ea150c+g560ea150c -> 5.3.4.post1.dev63+g560ea150c+g560ea150c
Arch aarch64 OS ubuntu 22.04
Python 3.11.15 Torch 2.10.0+cpu
FlagTree 0.6.0+ascend3.5 Triton 3.5.1
Vendor ascend Device npu

Test Result

Accuracy Result

Summary

Case Status Duration Total Passed Skipped Failed
Before Passed 38.67 sec 18 18 0 0
After Passed 67.31 sec 18 18 0 0

Performance Result

Summary

Case Status Duration Reason
Before Passed 56.05 seconds All tests completed.
After Passed 62.99 seconds All tests completed.

Benchmark data

Dtype Speedup Base Gems Shape
fp16 0.110 -> 0.111 - - -
0.174 -> 0.172 3.420 -> 3.415 19.640 -> 19.888 [[1073741824]]
0.068 -> 0.061 0.016 -> 0.016 0.231 -> 0.262 [[64,64]]
0.065 -> 0.074 0.016 -> 0.016 0.249 -> 0.220 [[4096,4096]]
0.069 -> 0.074 0.016 -> 0.016 0.229 -> 0.218 [[64,512,512]]
0.175 -> 0.175 3.412 -> 3.413 19.548 -> 19.490 [[1024,1024,1024]]
fp32 0.259 -> 0.259 - - -
0.309 -> 0.309 6.775 -> 6.784 21.908 -> 21.941 [[1073741824]]
0.068 -> 0.062 0.015 -> 0.016 0.226 -> 0.260 [[64,64]]
0.308 -> 0.306 0.092 -> 0.089 0.299 -> 0.291 [[4096,4096]]
0.302 -> 0.306 0.090 -> 0.087 0.297 -> 0.286 [[64,512,512]]
0.308 -> 0.310 6.790 -> 6.785 22.022 -> 21.855 [[1024,1024,1024]]
bf16 0.154 -> 0.151 - - -
0.259 -> 0.260 3.474 -> 3.475 13.391 -> 13.354 [[1073741824]]
0.066 -> 0.062 0.016 -> 0.016 0.237 -> 0.261 [[64,64]]
0.096 -> 0.090 0.022 -> 0.019 0.227 -> 0.210 [[4096,4096]]
0.093 -> 0.090 0.021 -> 0.019 0.227 -> 0.212 [[64,512,512]]
0.253 -> 0.254 3.475 -> 3.469 13.735 -> 13.659 [[1024,1024,1024]]

📎 Download the full log here

@huangyiqun
huangyiqun merged commit bfeca79 into master Aug 14, 2026
17 checks passed
@huangyiqun
huangyiqun deleted the fix_do_bench_npu_input_20260813 branch August 14, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@huangyiqun huangyiqun huangyiqun approved these changes
@0x45f 0x45f Awaiting requested review from 0x45f 0x45f is a code owner
@103yiran 103yiran Awaiting requested review from 103yiran 103yiran is a code owner
@douxetpur douxetpur Awaiting requested review from douxetpur douxetpur is a code owner
@bin913 bin913 Awaiting requested review from bin913 bin913 is a code owner
@w1120029931-bit w1120029931-bit Awaiting requested review from w1120029931-bit w1120029931-bit is a code owner
@hellojack163 hellojack163 Awaiting requested review from hellojack163 hellojack163 is a code owner
@Caeruleann Caeruleann Awaiting requested review from Caeruleann Caeruleann is a code owner
@tengqm tengqm Awaiting requested review from tengqm

Assignees

No one assigned

Labels

benchmark size/XSmall tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zhzhcookie @0x45f @huangyiqun